Measuring Perceptions and Preferences for Meritocracy
Juan Carlos Castillo, Julio Iturra, Luis Maldonado, Francisco Meneses & Jorge Atria
Study 1: Measuring meritocracy with international secondary data
Data
The data used in the analysis corresponds to the last available wave of the social inequality module from the International Social Survey Programme (ISSP), which is the most specialized international comparative survey in perceptions, attitudes and beliefs about inequality related issues. This wave corresponds to the year 2009 and covers attitudes towards a series of topics dealing with social inequality across 41 countries and 56021 individuals. Although there is data available from this module for previous waves (1987, 1992 and 1999), unfortunately there are several variables that are important for this research that were not included, particularly in 1987 and 1999, reason why we only use the 2009 wave.
Variables
There are a series of indicators in the ISSP survey that in the following we attempt to classify in our meritocracy measurement scheme. However, it is important to mention that the items were not originally thought as a specific measure of meritocracy. Still, many of the items have been used in previous research on meritocracy (and certainly will keep being used in the future) and before proposing a new measure we consider to give a first test to our proposed conceptualization using available data that not only covers several countries but also is validated by the research community.
The variables are presented below organized in perceptions and preferences:
- Perception of meritocracy/non-meritocracy: for operationalizing perceptions the closest set of ISSP’s indicators comes from the question asking about perceptions for opportunities to get ahead, which are usually considered as “meritocratic beliefs” in previous research. The general heading of the battery is: “To begin we have some questions about opportunities for getting ahead. Please tick one box for each of these to show how important you think it is for getting ahead in life.” This is followed by a list of statements to be answered in a scale with the following levels numbered from 1 to 5: essential, very important, fairly important, not very important, not important at all.
The classification is based on criteria of internal motivation (meritocratic) and structural constraints (non-meritocratic). There were two items from the battery that were excluded from the analysis as they would not fit into the classification. The first one was “having good education yourself”, since it was not clear whether this is could be due to individual motivation or system opportunities, and the second was “giving bribes”, as introduced elements of criminality that were beyond a non-meritocratic perception.
- Preferences for meritocracy-non meritocracy: for operationalizing normative preferences we used a list of items related to reasons for pay. The ISSP question was: In deciding how much people ought to earn, how important should each of these things be, in your opinion, rated in the same essential-non important at all scale as the questions for meritocratic preferences.
Table 1 below summarise the items classified according to the proposal based on perception and preference regarding to the meritocratic/non-meritocratic dimensions:
| Component | Dimensions | Item |
|---|---|---|
| Perception | Meritocratic | How important is having ambition? |
| How important is hard work? | ||
| Non-meritocratic | How important is coming from a wealthy family? | |
| How important is a person’s race? | ||
| How important is being born a man or a woman? | ||
| How important is knowing the right people | ||
| How important is having political connections | ||
| Preference | Meritocratic | How well he or she does the job? |
| How hard he or she works at the job? | ||
| Non-meritocratic | What is needed to support a family? | |
| Whether the person has children to support? |
Methods
The estimation was performed using Confirmatory Factor Analysis (CFA). CFA was conducted using the lavaan R package (version 0.6-3; Rosseel, 2020), with diagonally weighted least squares (DWLS) estimation due to the items’ ordinal level of measurement (Kline, 2016; Rosseel, 2020). As recommended by Brown (2008), we assessed model fit by jointly considering the comparative fit index and Tucker-Lewis Index (CFI and TLI; acceptable fit > 0.95), Root of the average squared residual approximation (RMSEA; acceptable fit < 0.08), Chi-square: (p-value; acceptable fit > 0.05, and Chi-square ratio > 3).
Results
Descriptive analyses
Figure 1 shows the distribution of responses across the indicators for each perception and preference in the meritocratic and non-meritocratic dimensions. On one side, we see that there is a certain degree of importance attributed to factors such as hard work and ambition in the process of getting ahead, concentrating 94.9% and 92.8% in the fairly important to essential categories.
By contrast, for the non-meritocratic aspects, parental background showed a more dispersed opinion, where 61.2% considers that coming from a wealthy family is important, and 76% considers that having educated parents is important. On the other hand, the background dimension shows that people’ opinions are generally in the opposite sense, in which 63.7% consider that a person’s racial origin is not very important or not important at all in the process of getting ahead in life, while 35% consider it important, and this is nearly identical regarding the importance of gender. Lastly, we can observe differences in the networks dimension, in which 85.7% consider it to be important to know the right people in the process of getting ahead in life, contrasting with 50% who consider political connections to be important.
Regarding normative preferences, we observe how well a person performs his or her job and amount of effort involved concentrates 98% and 96.9% of the responses in the categories that attribute importance to these characteristics when evaluating the pay that people deserve. Finally, in relation to structural constrains, 83% attribute importance to the fact that supporting a family should be a relevant factor with respect to the pay obtained, while having children to support translates into 76.4% of importance in this dimension.
Figure 1: Distribution of responses in the ISSP meritocracy items
In terms of the association between the indicators, Figure 2 shows the polychoric correlation matrix. Firstly, we see that the correlations between the indicators are consistent with the structure of the initially proposed structure of dimensions distinguishing between the perception and preferences of meritocratic and non-meritocratic factors. In general, the associations are positive between the perception and preference dimensions, such as the importance of hard work to get ahead and the importance of how hard the work is (\(r\)=0.34), however, they tend to be weak. A further aspect to consider is the relatively low correlation between dimensions concerning meritocratic and non-meritocratic aspects, (e.g. importance of hard work with educated parents, \(r\)=0.15). However, with respect to the non-meritocratic perception indicators, we can see that the associations demonstrate dimensionality, however, it is not entirely clear. In this regard, the indicators referring to parental characteristics have a positive and stronger association (\(r\)=0.56), but this association is much weaker in comparison to the other indicators of Non-meritocratic perception (e.g. Wealthy family and Gender, \(r\)=0.01). The same holds true for the Gender and Race indicators (\(r\)=0.6), and also applies to both the indicators of knowing the right people and having political connections (\(r\)=0.55). On this point, we will return to it in the next section.
Figure 2: Perceptions and preferences for ISSP meritocracy items polychoric correlations
Confirmatory Factor Analysis
This section presents the results of the confirmatory factor analysis. First, a four-dimensional model for meritocratic and non-meritocratic perception and preferences is estimated, considering all indicators of the non-meritocratic perception dimension as a single latent variable. Then, we estimate a model that follows the proposed four latent variable measurement approach, however, the non-meritocratic perception dimension is assumed as a second-order factor for the first order latent variables parental attributes, racial and gender background and networks.
| Model | \(N\) | Estimator | \(\chi^2\) | df | CFI | TLI | RMSEA |
|---|---|---|---|---|---|---|---|
| First order | 46594 | DWLS | 21308.535 | 48 | 0.959 | 0.944 | 0.098 |
| Second order | 46594 | DWLS | 4472.369 | 45 | 0.992 | 0.988 | 0.046 |
Table 2 shows the four-factor model fit allowing comparison of first and second-order factors. The model restricted to four first-order factor dimensions shows a regular fit (CFI=0.959, TLI=0.944, RMSEA=0.098, \(\chi^2\)(df=48)= 21308.535; more detail in Table 3) in contrast to the model considering a second-order factor for non-meritocratic perception that has a slightly better fit based on the standard fit measures (CFI=0.959, TLI=0.944, RMSEA=0.098, \(\chi^2\)(df=48)= 21308.535).
Figure 3: Confirmatory factor analysis of the ISSP indicators of Perceptions and Preferences for Meritocracy
Figure 3 shows the overall results for this model, evidencing an improvement in factor loadings as well as in the overall fit of the measurement model. On one hand, we see that perceptions are correlated with preferences, but this correlation is stronger for the meritocratic dimension of perception and preference (\(r\)=0.46) than for the non-meritocratic (\(r\)=0.27), which is much stronger when looking at the correlations between meritocratic and non-meritocratic, both in perception (\(r\)=0.04) and preference (\(r\)=0.13). These results show that, on one hand, the positive correlation between perception indicates that the two are not completely opposite poles of a continuum (\(r\)=0.30), which is reflected in the correlation between preferences (\(r\)=0.25). Furthermore, it is shown that the indicators of non-meritocratic perception refer to three other sub-dimensions and cannot be completely attributed to a an entirely common factor as has been employed in previous research.
Appendix
| Meritocratic | Non-meritocratic | Meritocratic | Non-meritocratic | |
|---|---|---|---|---|
| Importance: hard work | 0.709 | |||
| Importance: having ambition | 0.726 | |||
| Importance: wealthy family | 0.694 | |||
| Importance: educated parents | 0.609 | |||
| Importance: race | 0.624 | |||
| Importance: gender | 0.591 | |||
| Importance: knowing people | 0.612 | |||
| Importance: political connections | 0.644 | |||
| Reasons for pay: well job | 0.760 | |||
| Reasons for pay: hard job | 0.910 | |||
| Reasons for pay: support family | 0.837 | |||
| Reasons for pay: has children | 0.918 | |||
| \(\chi^2\text{(df)}\) | 21308.5(48) | |||
| \(\text{CFI}\) | 0.959 | |||
| \(\text{TLI}\) | 0.944 | |||
| \(\text{RMSEA}\) | 0.098 | |||
| \(N\) | 46594 | |||
| Note: Standardized factor loadings using DWLS estimator ; CFI = Comparative fit index ;TLI = Tucker-Lewis index; RMSEA = Root mean square error of approximation |